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(54) Title: VIDEO IMAGING SYSTEMS 

(57) Abstract 

Flow of traffic through an area is monitored by 
producing a video image of the area using a camera (10), 
dividing the video image into a matrix of pixels (23), 
sampling the pixels (23) of the whole video image at low 
resolution to detect changes in the pixels (23) which will 
indicate an object of interest and sampling the pixels of a 
local area (25) of the video image centred on said object of 
interest, at high resolution to detect the size of the object of 
interest and its direction of movement. 
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VIDEO IMAGING SYSTEMS 

The present invention relates to video imaging systems and in particular 
to a method and apparatus for monitoring the flow of traffic through an 
area. 

5 Video imaging systems have been proposed for monitoring the flow of 
pedestrians entering or leaving a pedestrian traffic area, for example the 
entrance to a retail outlet or shopping precinct. Such systems comprise a 
video imager, for example a CCD camera which is positioned above the 
area to provide a view of the traffic area, the images of the video imager 
10 being analyzed by suitable processing equipment, to count the number of 
pedestrians traversing the area and to detect the direction of movement 
of the pedestrians. In order to reduce the processing capacity required, it 
has been proposed in accordance with WO94/27408, to restrict 
processing of the video image to only a narrow band of the traffic area. 

15 This system may be tailored to accommodate, for example, obstructions 
to the flow of traffic. However, once the system has been set up, it is 
inflexible and would require resetting, if the traffic area is altered by the 
introduction, removal or relocation of obstructions. Furthermore, such 
systems can only measure bidirectional flow of traffic. If such systems 

20 were to be used at an intersection, several systems would consequently 
be required, one at each entrance to the intersection, in order to monitor 
movement of the pedestrians through the intersection. 

The present invention provides a video imaging system which is flexible 
and will permit the monitoring of omni-directional traffic flow. 

25 In accordance with one aspect of the present invention a method of 
monitoring the flow of traffic through an area is characterised by; 
producing a video image of the area; dividing the video image into a 
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matrix of pixels; sampling the pixels of the whole video image at low 
resolution to detect an object of interest; and sampling the pixels of a 
local area of the video image centred on said object of interest, at high 
resolution. 

In accordance with a further aspect of the present invention, a video 
imaging system comprises a video imager for producing a video image of 
an area and processing means for analyzing the video image, said 
processing means including; means for dividing the video image into a 
number of pixels; means for sampling pixels at low resolution to detect 
an object of interest; and means for sampling the pixels of a localised 
area centred on said object of interest, at high resolution. 

In accordance with the present invention, the whole of the video image is 
scanned at low resolution, say one in every twenty pixels in both 
directions, and only if an object of interest is located at low resolution, is 
a localised area corresponding to the detected object of interest scanned 
at higher resolution, thereby saving on the processing capacity required. 

In this manner, when an object of interest is located, it may be tracked 
across the area covered by the video image, so that omni-directional flow 
of traffic across the area may be monitored. A single video imager may 
consequently, for example, be used to monitor flow of traffic at an 
intersection. 

According to a preferred embodiment of the invention a series of time 
separated video images of the area are produced, the pixels of one video 
image being compared with the corresponding pixels of a preceding video 
image or images, so that an object of interest moving across the area may 
be detected by a change in the pixel. 

Furthermore, the local area scanned at high resolution may be sized to the 
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object of interest, so as to provide discrimination between different 
objects of interest. For example, the local area scanned at high resolution 
will differ depending on whether the object of interest is a child, adult or 
pedestrian pushing a trolley or a family group. The sizing of the local area 
5 scanned at high resolution, will also allow for discrimination between 

individual pedestrians and closely grouped pedestrians. Corrections may 
furthermore be made to the sized areas to accommodate variations due, 
for example, to perspective or fish-eye lens effects. Alternatively the 
changing shape of an object from frame to frame may be used to identify 
10 the object. For example, for a pedestrian the stride pattern may be used 
to distinguish between an adult and a child, while inanimate objects such 
as trollies will produce a regular pattern. 

By continuously scanning the whole traffic area, albeit at low resolution, 
the system will automatically accommodate changes in traffic flow paths 
1 5 when, for example, obstacles are introduced or relocated in the traffic 



area. 



20 



In accordance with a further aspect of the present invention, a video 
imaging system comprises a video imager for providing a video image of 
an area and processing means for analyzing the video image, said 
processing means comprising means for dividing the video image into one 
or more areas of interest, the processing means analyzing each area of 
interest individually to detect the rate of flow and direction of movement 
of objects of interest through each area of interest. 



25 



In accordance with this further aspect of the invention, a single video 
imager may be used to monitor individually a plurality of areas of interest 



in a given area. For example, the areas of interest may represent the 
entrances to or exists from a lift, escalators and stairs. A single video 
imager may thus be used to keep separate counts of pedestrians using 
the lift, escalators and stairs, whereas hitherto separate means would 
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have been required to monitor each area. Alternatively, the areas of 
interest may represent different counters of a retail outlet. 

The invention is now described, by way of example only, with reference 
to the accompanying drawings, in which:- 

Figure 1 illustrates diagrammatically, a video imaging system for 
monitoring the flow of pedestrians through a traffic area, in accordance 
with the present invention; 

Figure 2 is a diagrammatic illustration showing the processing of the 
video image obtained by the system illustrated in Figure 1 ; 

Figure 3 illustrates diagrammatically a video imaging system similar to 
that illustrated in Figure 1, installed at a pedestrian intersection; 

Figure 4 illustrates diagrammatically an alternative application of the video 
imaging system in accordance with the present invention; and 

Figure 5 illustrates diagrammatically a further embodiment of the present 
invention. 

As illustrated in Figure 1 , a video imaging system for monitoring the flow 
of customers in, for example supermarkets, comprises a monochrome or 
colour CCD camera 10 which is mounted over an isle 14 of the 
supermarket. Cabinets 16 are located on either side of the isle 14. The 
camera 10 is mounted to look vertically down on the area 12 which may, 
for example, be 10 metres square. 

The camera 10 is connected to a remote processing means 20 which 
comprises an analogue to digital converter 21 which converts the 
analogue signal from the camera 10 into a digital signal representing the 
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video image. The video image is divided into pixels, each representing a 
tiny dot on the video image. The digital signal includes a digital value 
corresponding to the shade of grey (for monochrome) or colour of each 
pixel. The video image of the area 12 comprises a matrix 512 x 512 
5 pixels. The camera 10 produces a new video image of the area 12 at a 
rate of 25 frames per second. 

The digital signals of each frame of the video image are passed to 
memory and analyzing means 22, which scans every twentieth pixel 23 
of the matrix in each direction and compares the digital value 

10 corresponding to the shade of grey or colour of each pixel 23 with that of 
the same pixel in the preceding frame. Alternatively the digital value of 
each pixel may be compared with a weighted average of the digital value 
of that pixel during an arbitrarily long period of say 255 frames. If there 
is a variation in the shade of grey or colour which is in excess of a 

15 threshold value, then a potential object of interest, that is a moving 

object, is detected. The threshold may vary at different light intensities 
and the ability to differentiate changes at high and low intensities may 
change. Threshold maps may be used to control the threshold for 
different areas of the image. 

20 In order to avoid the false detection of objects of interest due to local 

fluctuations in light, upon detection of a potential object of interest, pixels 
adjacent to the pixel 23 indicating a potential object of interest are 
scanned and only if there is a change in the shade of grey or colour above 
the threshold value, of at least one of these adjacent pixels, is the object 

25 of interest confirmed. Furthermore, in order to avoid the false detection 
of object of interest due to variations in light over the area 1 2 as a whole, 
the variation in shade of grey or colour of the pixel 23 identifying the 
potential object of interest, is compared with variations of pixels 23 in 
other parts of the area 1 2 which include, for example, pixels 23 in parts 

30 of the area 12 in which the display cabinets 16 are located and cannot 
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possibly include objects of interest. 

Upon confirmation of an object of interest, the analyzing means then 
scans all the pixels in a local area 25 which is centred on the pixel 23 
identifying the object of interest. The local area 25 may initially be of a 
nominal size corresponding to a typical adult pedestrian. The size of the 
local area 25 may however be increased or reduced until the local area 25 
is sized to the object of interest. The local area 25 is also moved until it 
covers the object of interest. That is, the local area 25 is adjusted until it 
covers all adjoining pixels in which there has been a change in the shade 
of grey or colour above the threshold value. 

The object of interest may be identified by the size of the local area 25 
which is eventually found to be required to cover the object of interest. 
The size and shape of the object of interest will change as the pedestrian 
moves and, for example, his arms and legs move relative to his body. 
Such variations may however be taken into account when identifying the 
object of interest by, for example, averaging the size over several frames 
of the video image. The processing means 20 is provided with memory 
means 24 in which look up tables are stored by which objects of interest 
in different size ranges would be identified as, for example, children, 
adults, adults pushing pushchairs or supermarket trolleys or closely 
grouped pedestrians. In the latter case, the overall size of the group may 
be used to provide a number in the group. The threshold value will 
discriminate between solid objects and shadows in the video image, thus 
avoiding false identification of the objects of interest. Where however 
intense shadows are formed by an intense light source, the threshold 
value may not be sufficient to discriminate between the solid object and 
shadow. The system may however be programmed with the position of 
such light sources, so that the location of a shadow may be predicted and 
discounted. 
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The local area 25 moves with the object of interest as it traverses the 
area 12 and thus the direction of movement of the object of interest may 
be monitored. 

With the system disclosed above, if an object of interest comes to rest in 
5 the area 12 it will merge into the background. In such cases, it is 

necessary to maintain a record of the position of the object of interest 
and where it came from, so that when the object begins to move again, 
its path across the area 12 may be detected. One method of achieving 
this is that once a pixel detects an object of interest, the digital value 
10 against which the digital value of the pixel is compared is frozen until the 
object of interest moves away from that pixel. This protection of 
stationary objects of interest will preferably be of limited duration so that 
new obstructions permanently introduced into the area 1 2 will eventually 
be permitted to blend into the background. 

1 5 The system disclosed above may consequently be used to monitor the 
number, type and direction of shoppers moving along the aisle 14. 
Furthermore, as the system covers the whole of the area 12, if the 
display cabinets 16 are moved so that customers may walk along the 
areas hitherto covered by the cabinets 1 6 and/or further display means 

20 are provided in the centre of the aisle 14, the system will automatically 
adapt to the new flow paths of the customers, without having to be 
reset. 

In the embodiment disclosed above, the video image a frame of which is 
illustrated in Figure 2 which typically represents an area 12, m x 10m, is 
25 divided into a matrix of 512 pixels x 512 pixels. Each pixel will 

consequently cover an area of about 2cm square. At course resolution 
when every twentieth pixel 23, in each direction, is scanned, the 
separation between the scanned pixels 23 will be about 40cms. This 
compares to an average width of a pedestrian of about 60cms. Scanning 
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at course resolution should consequently detect any pedestrians entering 
the area 12. However, the resolution at which the course scanning is 
carried out may be increased or decreased as required, depending on the 
area 1 2 that is covered by the video image, the relative size of the objects 
5 of interest and the number of pixels into which the image is divided. 

Scanning at course resolution will however preferably involve scanning 
between 5% and 10% of the pixels evenly over the whole area of the 
video image. 

Furthermore, in accordance with the above embodiment, pedestrians may 
10 move across the area 12 at speeds up to about 3m/sec. Assuming that a 
typical pedestrian is about 30cm from front to back, each pixel traversed 
by the pedestrian would cover the pedestrian for about 0.1 seconds, that 
is at least two frames at a rate of 25 frames/second. Moreover, the 
pedestrian will progress across the area 12 by up to about 6 pixels per 
1 5 frame. The direction and path of movement of the pedestrian across the 
area 12 may consequently be tracked frame by frame. 

In order to achieve real time processing of the video image, it is necessary 
to detect, confirm, size and position an object of interest between 
successive frames. Once an object of interest has been detected, sized 
20 and positioned, the size and position in one frame may be used to predict 
the size and position in the next frame, thereby reducing the amount of 
processing required in subsequent frames, to track movement of the 
object of interest across the area 12. 

The analyzer 22 is capable of scanning a plurality of local areas 25 at the 
25 same time, so that a plurality of objects of interest transversing area 1 2 
may be tracked simultaneously. 

As illustrated in Figure 3, a video imaging system of the type described 
above may be used at an intersection 30 where two pedestrian 
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thoroughfares 31, 32 intersect. With this system, as movement of the 
pedestrians is monitored over the whole of the intersection 30, the 
system is capable of monitoring pedestrians turning right or left as well as 
going straight across the intersection 30. 

5 In the embodiment illustrated in Figure 4, the camera 40 is positioned so 
that it looks horizontally and downwardly along the main aisle 42 of a 
supermarket. The system can thus monitor pedestrians moving along the 
main aisle 42 and also turning off the main aisle 42 into one of several 
aisles 44 leading from the main aisle 42. In this embodiment in order to 
10 accommodate perspective effects, the local areas 45 which are scanned 
at high resolution after an object of interest has been identified at low 
resolution, may be scaled. 

The embodiment illustrated in Figure 5 is similar to the system illustrated 
in Figure 1 and the same reference numerals are used for corresponding 

1 5 components. In the embodiment illustrated in Figure 5, the overall area 
1 2 covered by the camera 1 0 is divided into several areas of interest, a 
first area of interest 52 corresponding to the entrance/exit from a lift 53; 
a second area of interest 54 corresponding to the entrance to an up 
escalator 55; a third area of interest 56 corresponding to the exit from a 

20 down escalator 57; and a fourth area of interest 58 corresponding to the 
entrance/exit from stairs 59. 

The processing unit 20 is capable of processing each of the areas of 
interest 52, 54, 56 and 58 independently, so that separate counts can be 
obtained for pedestrians entering or exiting the lift 53, going up escalator 
25 55, coming down escalator 57 and going up or down the stairs 59. 

The areas of interest 52, 54, 56 and 58 are defined by the software of 
the system and may be set by the operator upon installation of the 
system. The areas of interest may furthermore be reset to accommodate 
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changes in the physical layout of the area 1 2, without changing the 
hardware of the video imaging system. Alternatively, the overall area 12 
covered by the camera 1 0 may be divided into areas of interest 
corresponding to, for example, different counters or display cabinets, so 
that movement of customers from one counter or display cabinet to 
another may be monitored. 

While in the above embodiments the camera 10 produces a new video 
image at a rate of 25 frames per second and the images are divided into a 
matrix of 512 x 512 pixels, the camera 10 may alternatively produce 
images at from 1 5 to 60 frames per second and the images may be 
divided into matrices of from 80 x 80 to 4000 x 4000 pixels. 

An additional video imager may be used to provide a stereoscopic image 
which would allow depth perception. This would assist in the 
identification of objects of interest and would also provide the capacity to 
see around objects which may be obscuring other objects of interest. 
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CLAIMS 

1 . A method of monitoring the flow of traffic through an area (12) 
characterised by; producing a video image of the area {1 2); dividing the 
video image into a matrix of pixels (23); sampling the pixels (23) of the 
whole video image at low resolution to detect an object of interest; and 
sampling the pixels of a local area (25, 45) of the video image centred on 
said object of interest, at high resolution. 

2. A method according to Claim 1 characterised in that the pixels (23) 
of one video image of an area (12) are compared with the corresponding 
pixels (23) of a video image or video images of the area (12) taken at a 
different time, so that changes in the pixels will identify an object of 
interest moving across the area (12). 

3. A method according to Claim 2 characterised in that video images 
are produced at a rate of 15 to 60 frames per second. 

4. A method according to Claim 2 or 3 characterised in that the video 
image is converted to a digital video image signal in which the video 
image is divided into a matrix of pixels (23), the digital video image signal 
containing a digital value for the shade of grey or colour of each pixel (23) 
in the matrix. 

5. A method according to Claim 4 characterised in that the shade of 
grey or colour of each pixel (23) of one video image is compared with the 
shade of grey or colour of the corresponding pixel (23) in a preceding 
video image and if the shade of grey or colour differs by more that a 
threshold value, an object of interest is identified. 

6. A method according to Claim 4 characterised in that the shade of 
grey or colour of each pixel (23) of one video image is compared to a 
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weighted average of the shade of grey or colour of that pixel (23) during 
an arbitrarily long period, and if the shade of grey or colour differs by 
more than a threshold value, an object of interest is identified. 

7. A method according to Claim 5 or 6 characterised in that changes 

5 in each pixel (23) are compared with changes in other pixels (23) over the 
area (12) as a whole, so that corrections may be made for changes in 
light over the area (1 2) as a whole. 

8. A method according to any one of the preceding claims 
characterised in that detection of an object of interest at low resolution is 

10 confirmed by scanning pixels (23) adjacent to the pixel (23) at which an 
object of interest is detected, prior to the scanning of a local area (25) 
centred on the object of interest. 

9. A method according to any one of the preceding claims 
characterised in that at low resolution, between 5% and 10% of the 

1 5 pixels (23) are scanned evenly over the whole of the video image. 

10. A method according to any one of the preceding claims 
characterised in that the local area (25) is sized to the object of interest. 

11. A method according to Claim 10 characterised in that after locating 
an object of interest by scanning at low resolution, a local area (25) of 

20 nominal size is centred on the object of interest and the local area (25) is 
scanned at high resolution, the local area (25) being gradually sized to the 
object of interest over several high resolution scans. 

12. A method according to Claim 10 or 11 characterised in that the 
object of interest is identified from the size of the local area (25). 

25 13. A method according to Claim 10 or 11 characterised in that the 
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object of interest is identified by variations in its shape from frame to 
frame. 

14. A method according to any one of the preceding claims 
characterised in that each video image is divided into a matrix of from 80 

5 x 80 to 4000 x 4000 pixels (23). 

15. A method according to any one of the preceding claims 
characterised in that the video image is divided into a plurality of areas of 
interest (52, 54, 56, 58), each area of interest (52, 54, 56, 58) being 
scannable independently. 

10 16. A method for monitoring the flow of traffic through an area 

characterised by producing a video image of the area (12), dividing the 
video image into a plurality of areas of interest (52, 54, 56, 58) and 
processing the video image to detect independently, objects of interest in 
each area of interest (52, 54, 56, 58). 

15 17. A method according to Claim 16 characterised in that a series of 
time separated video images are produced, the video images being 
processed to detect the rate of flow and direction of movement of objects 
of interest, through each area of interest (52, 54, 56, 58). 

18. A video imaging system characterised by a video imager (10; 40) 
20 for producing a video image of an area (12; 30; 42) and processing 

means for analyzing the video image, said processing means (20) 
including means (21) for dividing the video image into a number of pixels 
(23), means (22) for sampling pixels (23) at low resolution to detect an 
object of interest; and means for sampling the pixels (23) of a localised 
25 area (25, 45), centred on said object of interest, at high resolution. 

19. A video imaging system according to Claim 18 characterised in that 
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the video imager (10; 40) is a monochrome or colour camera. 

20. A video imaging system according to Claim 18 or 19 characterised 
in that the video imager (10; 40) is disposed vertically, horizontally or in 
between the vertical and horizontal relative to the area (12; 30; 42). 

21 . A video imaging system according to any one of Claims 18 to 20 
characterised in that an additional video imager is provided to give a 
stereoscopic image of the area (42). 
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